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ABSTRACT 

In April 2006, the Michigan State Board of Education and Michigan Legislatures adopted a rigorous package of high 
school graduation reguirements, one of which made Michigan the first state that incorporated an online learning 
graduation reguirement into the K-12 curriculum. All Michigan's students entering high school during 2008-2009 school 
year were required to complete online learning during their course of high school studies in order to graduate. Michigan 
Virtual School helped the schools in Michigan to fulfill this requirement by developing a 20-hour online learning course 
called "Career Forward". In December 2008, the Michigan Virtual University provided the National Repository of Online 
Courses access to the CareerForward course content, allowing students from anywhere in the United States, the ability to 
access CareerForward free of charge. This evaluation study was conducted to provide Michigan Virtual School with 
information to improve the design and delivery of fhe Career Forward course, in order fo improve fhe learning 
experiences of the future students and to improve the overall efficiency of the course. Analysis of data from this research 
indicated that, CareerForward in its current format had very little impact on student attitude towards career planning. 
Recommendations for changes in design and delivery options of fhe course for future offerings are suggested in order to 
make the course more effective and to meet its objectives. 

Keywords: K-12 Online Learning, Virtual Schooling, Cyber Schooling, Graduation Requirement, Online Course, Career 
Education. 


INTRODUCTION 

Fulton (2002) predicted that by 2006, a majority of 
American high school students would have taken atleast 
one online course prior to graduation, Though this 
prediction at that time seemed implausible in reality, 
subsequent developments in K-12 Education made this 
prediction realistic. In the Michigan Merit Curriculum 
Guidelines: Online Experience, the State of Michigan 
outlined their decision to become the first state in the 
United States to require that, all students take at least one 
course online prior to high school graduation (Department 
of Education, 2006). This state mandate for online learning 
provided the basis for a dramatic increase in the number of 


students enrolled in virtual school courses in Michigan only 
to be followed by the other states across the nation 
(Barbour & Reeves, 2009). 

Cavanaugh, Barbour and Clark (2009) in their review of 
literature indicated that most of the research works 
published in relevance to K-l 2 online education focused 
on the experiences of the virtual school teacher or 
administrator and the majority of their reviewed literature 
described the experiences of these individuals. Another 
study by Barbour and Reeves (2009) suggested that, the 
body of published literature on K-l 2 online learning could 
be classified as falling under two categories: (i) potential 
benefits of K-12 online learning and (ii) the challenges 
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facing K-12 online learning. Barbour and Reeves also 
mentioned that, there was a "deficit of rigorous reviews of 
the literature related to virtual schools" (p. 402). Barbour 
(2010) discussed how much amount of published research 
on K-l 2 online education is limited and also some of the 
published researches conducted on K-12 online 
education suffer from methodological flaws or have 
attempted to investigate beyond the scope of the 
researcher's inquiry. 

In the following sections of this research article, the author 
discuss in details the Online Learning Requirement in 
Michigan and how the Michigan Virtual School helped the 
schools in Michigan to fulfill this requirement. They also 
discussed Career Forward as an online learning program, 
how it was offered, how it was accessed by students, the 
modules of the program and the five step learning cycle in 
Career Forward modules. Following this, the methodology 
of research for this evaluation study, were proposed and 
they analyze the data from the study to discuss findings and 
draw conclusions about the effectiveness of Career 
Forward as a career preparatory course. The authors end 
this article with the discussion of the limitations of this study 
and its implications. 

The Michigan Online Learning Requirement 

The Michigan Virtual School (MVS) is a state sponsored 
virtual school designed to provide online learning 
opportunities to students and teachers in the state of 
Michigan (Clark, 2001). In April 2006, the Michigan State 
Board of Education and Michigan Legislatures adopted a 
rigorous package of high school graduation requirements, 
one of which made Michigan the first state that 
incorporated an online learning graduation requirement in 
the K-12 curriculum. All Michigan's students entering high 
school during 2008-2009 school year were required to 
complete online learning during their course of high school 
studies in order to graduate. This requirement could be met 
by completing a completely online course or by blended 
instruction format with 20 hours of online instruction within 
an in-class course. To help Michigan's schools meet this 
requirement, the Michigan Virtual School developed a 20- 
hour online learning course called CareerForward. In 
December 2008, the Michigan Virtual University (MVU) 


provided the National Repository of Online Courses (NROC) 
access to the CareerForward course content, allowing 
students from anywhere in the United States the ability to 
freely access CareerForward. 

What is CareerForward? 

CareerForward (from http://nroc.careerforward.org/) is an 
online learning program created to assist middle and high 
school students with planning future career paths, and 
developing an understanding of what it takes to achieve 
that desired career. Developed through a partnership 
between Michigan Department of Education and MVU, 
and with funding provided by the Microsoft Corporation, 
this free online learning experience was designed to be a 
self-contained motivational tool that allowed students to 
explore 21 st century career possibilities, and at the same 
time students would also meet the mandatory online 
learning graduation requirements of the state. This 4-6 
week long course address questions like: 

• "What am I going to do with my life?" 

• "What is the working world like?" and 

• "How do I match my interests with work?" 

The course also uses a variety of multimedia and online 
resources to address the content, allowing Michigan 
students to meet the new online learning requirement for 
graduation. 

CareerForward can be accessed or downloaded in three 
different learning environments: 

(1) MVU-hosted web-based version; 

(2) MVU-hosted Blackboard CMS (Course Management 
System) version, and 

(3) School-hosted Blackboard or Moodle versions (Figure 1). 

Schools are required to register their students and teachers 
to the CareerForward program. This registration process 
allows Microsoft to keep tract of the users enrolled in the 



Figure 1. The Modes of Delivery of CareerForward 
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CareerForward program even though, it is offered free to all 
users, Schools then decide the mode of delivery of the 
course to their students, Some schools chose to integrate it 
into Business and Computer courses, while others 
implement it as a stand-alone course. CareerForward can 
be also be companioned with a variety of online career 
planning tools (e.g. Career Cruising and Career Explorer), 
which allow students the opportunity to assess their career 
interests, explore career options, and create an 
Educational development plan (Barbour, 2009). 

CareerForward is divided into four modules that students 
complete, each of which guides them through a five step 
learning cycle (Figure 2); beginning with a scenario or 
challenge. The student is given framing questions for the 
module, and they must provide their initial thoughts. Next, 
the student reviews resources, that come in the form of 
online videos and reading material. After viewing the 
resources, the student is asked to complete the same 
framing questions to see if their initial thoughts have 
changed. The final step of the cycle is completing a 
packet of activities related to the module's content. 

Methodology 

This evaluation study was conducted to provide Michigan 
Virtual School with reliable data and information required to 
improve the design and delivery of the CareerForward 
course, in order to improve the learning experiences of the 
future students and to improve the overall efficiency of the 
course. This led to the following research questions: 


Step!: Students are 
presented with a Scenario 
or Challenge in Module (i, i= 
1 to 4) 


StepV :Students complete a 
packet of activities related 
to the Module(i)'s content 


Figure 2. The Five Step Learning Cycle in 
each CareerForward Module 


1. What impact does taking the CareerForward course 
have on student attitudes towards career planning? 

2. What are the student experiences of the 
CareerForward course? 

3. How would students improve the CareerForward 
course? 

Data Collection 

Two surveys, designed to be taken as a pre-course survey 
and a post-course survey were used by the MVS to gather 
data from students who participated. 

The pre-course survey was used to gather information on 
demographic data related to gender, grade level, why the 
students were taking CareerForward, how the 
CareerForward course was structured (whether as a 
standalone course or as a part of another course) and the 
medium of delivery. The pre-course survey was also used to 
collect student ratings of 14 statements related to career 
planning, based on Likert scale. The post-course survey 
contained all of the items from the pre-course survey and 
some additional items. Students were asked where they 
completed most of their CareerForward course; also 
included where survey items that collected students' 
impressions of the CareerForward course; and finally in 
open-ended questions, students were also asked to 
comment on the changes they would suggest for the 
CareerForward course as the best and the most difficult 
things about the course. Both instruments were designed to 
the MVS prior to the evaluation request made of the 
researchers. 

A total of 3899 students participated in the pre-course 
survey and 382 students completed the post-course 
survey. The MVS provided the researcher with de-identified 
data. As the researchers were unable to link the pre-course 
and post-course data, the quantitative data were 
compared based on the overall means on each item. The 
open-ended qualitative data was analyzed using a 
method that utilized MS Word (Ruona, 2005). 

Results 

A total of 49.4% of the participants who completed the pre¬ 
course survey were female, while 55.2% of the participants 
who completed the post-course survey were also female. 


Stepll Students provide 
their initial thoughts to the 
framing questions for the 
Module(i) 


StepIV : Students complete 
the same framing questions 
to see if they deviated from 
their initial responses 


Steplll: Students review 
resources in the form of 
online videos and reading 
materials 
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Grade nine students made up almost half of the pre¬ 
course survey sample (46.6%), while it reduced to less than 
a third of the post-course survey (29.1%). Grade eight 
students more than doubled from 13.4 to 30.9% from pre¬ 
course to post-course survey. Proportion of grade ten 
students also increased (i.e., 18.3% to 25.9%), while grade 
seven and eleven remained fairly consistent (i.e., 2.1% to 
0.3% and 10.8% to 9.4% respectively), Grade twelve 
student participants reduced to half as much from pre¬ 
course to post-course survey (8.7% to 4.5% in post-course). 

Table 1 Indicates the student feedback on why students 
took CareerForward(as indicated in the pre-course and the 
post- course survey data). 


Why did you take Career Forward? 


Pre-Course 

Pre-Course 

CareerForward was mandatory in school 

88.4% 

93.7% 

CareerForward was a part of Computer 
Course 


48.3% 

53.1% 

CareerForward was a standalone course 

9.3% 

22.8% 

CareerForward as a part of Career 
Planning course 


22.7% 

11.3% 

CareerForward as a part of Business course 

9.6% 

6.3% 

Table 1. Why Students Took Career Forward? 

Statement Pre-Course 

x Post-Course 

x Difference 

1 think about what I'll do after 
high school. 

4.23 

4.26 

0.03 

1 know what I'll do once 1 graduate 
from high school. 

3.73 

3.62 

-0.11 

1 can shape my career path. 

3.96 

4.06 

0.10 

1 plan to get help from others to 
get the jobs 1 want. 

3.64 

3.75 

0.11 

I'll have more than one job in 
my future. 

3.48 

3.59 

0.11 

My career will be affected by the 
global economy. 

3.20 

3.40 

0.20 

Starting my own business sounds 
like fun. 

3.46 

3.40 

-0.06 

I've started to plan my career path. 

3.70 

3.75 

0.05 

1 feel comfortable using computers 
and the Internet. 

4.22 

4.21 

-0.01 

The classes 1 take in high school will 
help me to achieve my career goals. 

3.93 

4.16 

0.19 

1 talk to my parents about my 
possible career plans. 

3.93 

3.91 

-0.02 

1 talk to other adults about my 
possible career plans. 

3.66 

3.66 

0 

1 talk to my friends about my 
possible career plans. 

3.65 

3.57 

-0.08 

1 explore the web for career or 
college Information. 

3.39 

3.53 

0.14 

Average 

3.727 

3.773 

0.046 


Table 2. Effects of CareerForward on Student Attitudes 


Data Analysis and Discussion 

In the following sections, the author discusses the findings 
from the data analysis based on the pre-course and post¬ 
course surveys. The findings are organized in the order of 
the research questions that guided this inquiry. 

1. What Impact does taking Careerforward Course have 
on Student Attitudes towards Career Planning? 

The impact of CareerForward on student attitudes towards 
career planning can be found by studying Table 2. 

The difference in students' average attitudes over the 
fourteen items as listed in Table 2 as a result of the 
CareerForward course is 0.046 which indicates that the 
course content had little positive significance on impacting 
students' attitude towards career planning. Table 2 data 
actually indicate five areas (the red cells) where student 
responses showed a decline as an impact from pre-course 
to post-course response. It is infer red from this trend that, 
the CareerForward course was in general ineffective in 
impacting student attitude towards their future career 
planning in a positive way. 

Table 2 indicate the largest positive difference between the 
pre-course and post-course surveys was for that statement, 
"My career will be affected by the global economy." After 
removing the participants who chose not to disclose their 
gender, it was observed that CareerForward was twice as 
effective in impacting attitudes of male students as female 
students (Table 3). 

The original difference in the pre and post course rating was 
0.20 for the category "My career will be affected by the 
global economy". Data was analyzed by gender after 
removing the participants who chose not to disclose 
gender. Data from Table 3 indicates that, CareerForward 
was twice as effective in impacting the attitudes of male 
students as females. 


Gender 

Pre-Course 

Pre-Course 

Difference 

Male 

3.16 (n= 1805) 

3.66 (n = 140) 

0.50 

Female 

3.27 (n= 1798) 

3.52 (n= 163) 

0.25 

Combined 

3.21 (n=3603) 

3.58 (n=303) 

0.27 

Original Total 

3.20 (n=3899) 

3.40 (n=382) 

0.20 


Table 3. My Career will be Affected by the 
Global Economy by Gender 
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Grade Level 

Average of difference in student attitudes 
(Post Course - Pre Course) 

Grade 7 

Data not considered (one response) 

Grade 8 

0.148 

Grade 9 

0.046 

Grade 10 

0.039 

Grade 11 

0.196 

Grade 12 

-0.175 

Overall 

0.064 

Table 4. Average difference in Student Attitude (by Grade) 

How CareerForward 
was offered 

Average of difference in student 
attitudes (Post Course-Pre Course) 

Entire Course 

0.031 

Career Planning 

0.130 

Business 

-0.027 

Computers 

0.069 

Another 

0.2 

Overall 

0.061 


Table 5. Average difference in Student Attitude 
by how CareerForward was offered 


Interesting observations were noted about the impact of 
CareerForward on student attitudes based upon their 
grade level. Grade seven had only one respondent for 
post-course survey, so that, the data was excluded. Table 4 
shows an average difference in the student attitude by 
grade. 

Table 5 indicates that, students who had taken 
CareerForward as a "part of a business course" had 
benefited the least from the course, in fact there was a 
negative impact of CareerForward towards student 
attitude, The most significant positive impact of change in 
attitude towards career planning due to the course was 
observed for students who had indicated they had taken 
CareerForward as a "part of another course". The survey 
was not designed in a way to record what other courses 
students were enrolled in with CareerForward. Positive gains 
were also recorded for students who had taken 
CareerForward as a part of career planning course. 
Students who had taken CareerForward as a standalone 
course also did not benefit much from the course. 

2. What are the Student Experiences of the 
CareerForward Course? 

Student experiences and satisfaction with CareerForward 
were collected using Likert scale from the data gathered 
by post-course survey items: 


• I enjoyed CareerForward. 

• I feel CareerForward was helpful. 

• I think this course was a powerful way to gain new 
information. 

Data analysis from these items in the post-course survey 
revealed that, the students did not enjoy the 
CareerForward course. 35.6% students who completed 
post-course survey strongly disagreed with the statement "I 
enjoy CareerForward". Also, almost 60% of the students or 6 
out of every 10 students either strongly disagreed or 
disagreed that, they enjoyed CareerForward. Almost half of 
the participants though agreed or strongly agreed that 
CareerForward was helpful and thought that 
CareerForward was a powerful way to acquire new 
information. Hence, it can be concluded that, though the 
students did not enjoy the CareerForward course, they 
found the information helpful and that the course was a 
powerful way to gain information. 

While there were no significant differences in student 
satisfaction level in using CareerForward based on gender, 
there were some differences observed based on grade 
levels of the population. As discussed earlier, while majority 
of the students did not enjoy the course, students in Grade 
eleven were the only group who rated CareerForward 
favorably on satisfaction with the course. Grade level 
analysis of data also indicated that, while most students 
found CareerForward helpful, it was most helpful to 
students in grades eight and eleven. Grade eleven 
students followed by grade eight students found the course 
a powerful source of information. The course was not 
helpful to students in grade twelve and students in grade 
ten and twelve did not find the course a powerful source to 
acquire new information. Hence, grades eight and eleven 
were the main beneficiaries of the course, while the course 
was least effective for students in grade ten and twelve. 

Finally, students who took CareerForward as a part of career 
planning course were more satisfied with the course than 
other students. Attitude data also indicated that, students 
who took CareerForward as a "part of another course" 
gained higher average gains in attitude due to the course. 
Similar results were observed with course satisfaction and 
students who had taken the course as "a part of another 
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course" reported positive experience with the course. 
Students who had taken CareerForward as an 
independent and standalone course reported most 
negatively on course satisfaction items. 

3. How would Students Improve the CareerForward 
Course? 

Three open-ended questions were included in the post¬ 
course survey to gather student perspectives on how 
learning experiences with CareerForward could be 
improved. The prompts used were: 

• if I could make one change in CareerForward it would 
be... 

• The best thing about CareerForward was... 

• The most difficult thing about CareerForward was... 

While there was no specific demographic trends observed 
in the open ended data, the authors present below some 
selected student perspectives that emerged as a result of 
the data analysis on the questions. 

Best Thing about CareerForward 
Three themes emerged from analysis of data in response to 
the open ended question on the Best thing about 
CareerForward. Students mentioned how the content of 
the course forced them to think about future career and life 
after graduation. For some high school students, this might 
have been the first time when they thought about future 
careers in a systematic way. The second theme was about 
how the use of videos in the course content helped the 
students gain information. The students provided feedback 
that the videos were helpful in conveying a lot of 
information in a compact way and it helped them receive 
useful information from actual individuals discussing their 
career choices. The third theme that emerged from the 
students 1 responses again related to the course content 
and how it helped the students relate careers to a specific 
skill set, and in a way to think and to identity careers that 
were suitable to their own skill set. There was a subgroup of 
students, approximately 10% of the participants, who 
responded that nothing was best about CareerForward. 

Most Difficult Thing About CareerForward 

Four themes emerged from the responses of the 

participating students for this question. The first thing that the 


students reported as most difficult was the volume of written 
work that CareerForward course included. Students also 
found some questions too vague, sometimes the instructor 
imposed minimum sentence or work requirements which 
made it more difficult for the students, as sometimes they 
were repeating the same or similar answer but related to 
different videos. The second issue that emerged focused 
on the relationship of the videos to the written work. The 
students felt that, the written work was not always aligned 
with the videos and in an effort to find information for the 
responses from the videos they had to watch those videos 
repeatedly. The third theme that emerged was the nature 
of the videos. As discussed in the previous section, while 
many students found the video content helpful and 
informative, students also complained that, the videos 
were lengthy, repetitive and boring. Another theme in this 
section that for some students was the material and course 
content for CareerForward was difficult to understand. It is 
possible that the students found the course difficult due to 
the large volume of written work or not always being able to 
find the necessary information from the videos right away; 
but the student responses were general and hence this 
connection cannot be drawn as an inference. 

One Change in CareerForward 

Three main themes emerged in this category from the 
open-ended responses of the students. The first theme 
related to the videos and the students suggested that, 
there should be a decrease in the number of videos that 
the students had to watch. The second theme also related 
to the videos used as instructional materials. Students are 
suggested to the use of shorter videos and also 
segmenting longer videos into shorter videos. Students are 
also suggested using varied methods like games and other 
interactive activities for instruction rather than using just 
videos as the main instructional materials. The third 
suggestion from the students related to the nature of the 
written work. Like the content of the videos students found 
the written work repetitive and boring. Students suggested 
varying the nature of activities, specifically incorporating 
more computer based activities to submit as required 
coursework. 
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Limitations 

This study had a number of limitations that could potentially 
affect the generalizability of the results. First, since all of the 
identifiable information had been removed from the data 
by the MVS, the researcher was unable to conduct any 
statistical analysis (e.g. t-tests, ANOVA, ANCOVA, regression 
etc.), which would have helped to determine the impact of 
CareerForward on changes in attitude by comparing a 
single students pre-course survey scores with post-course 
survey scores. Hence comparison of means was based on 
finding differences that would represent between 5% and 
10% of the mean score. 

Another major limitation was, the sample for the post¬ 
course survey was approximately 10% of the pre-course 
survey sample. This disproportional difference in sample 
sizes would have limited the ability to draw conclusions 
even if the data had been identified and the evaluator 
could have conducted a more sophisticated statistical 
analysis. The smaller sample size in case of the post-course 
survey was problematic while analyzing the data based 
upon demographics. For example, there was a single 
seventh grade student who completed the post-course 
survey, also a single student who took the course as a part 
of a World Studies course. There were only four students who 
found the course on their own. Finally, there were no 
students who took the course using a CD-ROM and only six 
students who took the course using something other than 
the Blackboard, Moodle, or a website. These low response 
rates for these characteristics, made it impossible for the 
researcher to include these variables as a part of the 
demographic analysis. 

Finally, approximately half of the students who indicated 
that, they took CareerForward using "a website" also 
indicated that, they took CareerForward using either 
"Blackboard" or "Moodle". This level of student 
misunderstanding between these of variables led the 
researcher to exclude "a website" in the analysis of the 
demographic data. 

Conclusions and Implications 

Overall, there was a little impact of CareerForward on 
student attitude towards career planning. Students did not 
enjoy the course though they found the course content 


helpful and CareerForward also enabled access to new 
and valuable information. Students also found videos an 
effective component in the course delivery and in fact 
indicated videos as the best part in the course. At the same 
time, students also reported that, the use of lengthy videos 
and written work throughout the course rendered the 
course, repetitive and boring. They suggested shorter 
videos and segmenting of longer videos into smaller 
segments, Students also wanted CareerForward to 
incorporate games and other computer based activities 
as additional ways to deliver content and student 
assignments. 

In terms of implications for practice, according to the data, 
the online career preparatory course content should be 
revised, so that the content contains fewer videos. Also, the 
length of videos should be shortened, with longer videos 
segmented into parts that can be independently viewed, 
Further, students suggested that, the inclusion of gaming 
and other interactive activities within the course content. 
Designers of such courses should carefully consider 
multiple pedagogically sound methods to deliver 
information and explore and incorporate varied formats of 
presenting information that appeals to the learners' 
preferences and styles while making the content 
instructional^ sound and effective. Finally, as 
CareerForward was found to have a little effect for grade 
twelve students and more effect for grades eight and 
eleven, administrators should consider whether career 
preparatory courses may be more effective for students 
who still have some time for exploring alternatives and 
decision making in career choices. 

There are three main areas for future research. First, more 
research is required to investigate the option "as a part of 
another course." Students who selected this option had 
higher than average scores in terms of effect of 
CareerForward on their attitudes towards career planning 
and their overall enjoyment with the course. Research is 
needed to determine which courses were considered 
under this category that complemented CareerForward 
better than other specific courses. Second, survey items 
should be designed more specific. Future investigations 
should provide clear distinctions between the options "a 
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website", "Blackboard" and "Moodle" which were 
confusing to the respondents for this study. This is an 
important implication since analysis of learning 
environment and platform might have yielded valuable 
information on the course delivery. Third, further exploration 
of usability and pilot testing of newer methods of content 
delivery and pedagogically sound activities are required to 
improve the course effectiveness. 
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